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ABSTRACT 


For  the  problem  max  (Z(S)  :  S  is  an  Independent  set  in  the  matroid  X} ,  it  is 
well-known  that  the  greedy  algorithm  finds  an  optimal  solution  when  Z  is  an  additive 
set  function  (Rado-Edmonds  theorem).  Fisher,  Nemhauser  and  Wolsey  have  shown  that, 
when  Z  is  a  nondecreasing  submodular  set  function  satisfying  Z  (0)  *  0 ,  the  greedy 
algorithm  finds  a  solution  with  value  at  least  half  the  optimum  value.  In  this  paper 
we  show  that  is  finds  a  solution  with  value  at  least  1/(1  +  Of)  times  the  optimum 
value,  where  O'  is  a  parameter  which  represents  the  "total  curvature"  of  Z.  This 
parameter  satisfies  0  <  »  <  1  and  a  *  0  if  and  only  if  the  set  function  Z  is  additive. 
Thus  the  theorems  of  Rado-Edmonds  and  Fisher-Nemhauser-Wolsey  are  both  contained  in 
the  bound  1/(1  +  ->).  We  show  that  this  bound  is  best  possible  in  terms  of  Of. 

Another  bound  which  generalizes  the  Rado-Edmonds  theorem  is  given  in  terms  of  a 
"greedy  curvature"  of  the  set  function.  Unlike  the  first  bound,  this  bound  can 
prove  the  optimality  of  the  greedy  algorithm  even  in  instances  where  Z  is  not  addi¬ 
tive.  A  third  bound,  in  terms  of  the  rank  and  the  girth  of  X,  unifies  and  general¬ 
izes  the  bounds  (e-l)/e  known  for  uniform  matroids  and  1/2  for  general  matroids.  We 
also  analyze  the  performance  of  the  greedy  algorithm  when  X  is  an  independence  sys¬ 
tem  instead  of  a  matroid.  Then  we  derive  two  bounds,  both  tight: 

ic 

The  first  omls  (1  -  (1  -  y/K)  ]/y  where  K  and  k  are  the  sizes  of  the  largest  and 
smallest  maximal  independent  sets  in  X  respectively;  the  second  one  is  1/ (p  +  <v) 
where  p  is  the  minimum  number  of  matroids  that  must  be  intersected  to  obtain  X. 

Kev  Words:  Combinatorial  Optimization,  Greedy  Algorithm,  Matroid,  Submodular  Set 
Function,  Worst-Case  Analysis,  Heuristic,  Tight  Bound. 
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1.  INTRODUCTION 


Many  problems  in  combinatorial  optimization  can  be  written  in  a  natural 
way  as 

(1.1)  max  (Z(S)  :  SEX) 

where  X  is  a  family  of  subsets  of  a  finite  set  N  and  Z  is  a  set  func¬ 
tion  defined  on  (S  C  N) . 

The  family  X  is  called  an  ind a pendene e  system  if 

(1.2)  Sex  and  T  c  S  **  T  €  X. 

The  sets  in  X  are  often  called  independent  sets.  If  furthermore 

(1.3)  S,  T  €  X  and  |T|+l=|s|»3j€S-T  such  that  Tu{j)  £  X, 

then  the  family  X  is  called  a  mairoid.  See  [12],  [10]. 

For  a  set  function  Z,  we  define  the  discrete  derivative  at  S  C  N 
in  direction  j  £  N  as  (S)  =  Z(S  U  { j })  -  Z(S).  The  set  function  Z  is 
said  to  be  submodular  if 

(1.4)  TCSCK*p.(T)>p.(S)  for  all  j  £  K-S. 

A  greedy  (or  steepest  ascent)  algorithm  comes  naturally  to  mind  when 
X  is  a  matroid  (or  an  independence  system)  and  when  Z  is  submodular. 

GREEDY  ALGORITHM  :  Start  with  the  empty  set.  Then  recursively  add  to  the 
current  solution  set  S  an  element  j  with  the  largest  discrete  derivative 
Pj(S)  among  all  j  £  N-S  such  that  S  U  {j}  £  X  and  Pj  (S)  >0.  Stop 
when  no  such  element  exists. 
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U’e  11 -known  examples  of  problems  fitting  the  framework  (1.1)  include 

(1*5)  the  problem  of  finding  a  maximum  weight  independent  set  in  a 
matroid  :  X  is  a  matroid  and  Z  is  additive  (i.e.,  o.(S)  =  p., 
a  constant  independent  of  S).  Then  the  greedy  algorithm  finds 
an  optimal  solution  (Rado-Edmonds  theorem)  [3];  a  common  applica¬ 
tion  occurs  when  the  independent  sets  are  the  forests  of  a  graph 

(9],  r 103. 

(1.6)  a  simple  plant  location  problem  [2]  :  X  is  a  uni  fair,  matroid 
(i.e.  X  =  {S  C  N  :  js|  <  K})  and  Z  is  a  nondecreasing  sub- 
modular  set  function  with  Z(0)  =  0.  Then  the  greedy  algorithm 
finds  a  solution  with  a  value  which  is  guaranteed  to  be  at  least 
(e-l)/e  times  the  optimum  value  [2],  [11],  where  e  is  the  base 
of  the  natural  logaritlims; 

(1.7)  the  problem  of  finding  a  set  of  maximum  weight  in  the  intersection 
of  two  matroids  :  X  is  one  of  the  two  matroids  and,  for  all 

S  c  N,  Z(S)  is  the  maximum  weight  over  all  sets  T  c  S  which  are 
independent  in  the  second  matroid.  Then  the  greedy  algoritljn 
guarantees  a  solution  within  50  %  of  the  optimum  [5]. 

These  are  three  examples  where  the  feasible  set  is  a  matroid.  Although 
the  hound  guaranteed  by  the  greedy  algorithm  is  different  in  each  case  we 

believe  that  these  results  can  be  unified.  For  example,  we  will  show  that  the 
bounds  (1.6)  and  (1.7)  are  the  two  extreme  values  of  a  bound  expressed  in  terms 
of  the  cardinalities  of  the  smallest  Infeasible  and  largest  feasible  sets. 

These  parameters  are  called  the  girth  and  the  rank  of  the  matroid  X  respectively. 
We  will  also  show  that  the  bounds  (1.5)  and  (1.7)  are  the  two  extreme  values  of 
a  bound  expressed  in  terms  of  a  parameter  reflecting  the  "total  curvature"  of 
the  function  Z  (see  definition  below) . 
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It  will  be  convenient  to  assume  that,  in  (1.1),  the  objective  function 
is  nondecreasing  and  satisfies  Z(0)  =  0.  (As  in  [11],  general  submodular 
set  functions  can  be  handled  by  using  an  appropriate  performance  measure; 
however,  with  the  above  assumption,  the  greedy  performance  will  simply  be 
given  as  a  percentage  of  the  optimum  value.)  Nondecreasing  submodular  set 
functions  such  that  Z(0)  =  0  are  subadditive  (i.e.  Z(S)  +  Z(T)  >  Z(S  u  T) 
VS,  T  C  N).  They  arise  in  location  theory  and  more  generally  in  economic 


once  a  set  S  of  actions  is  already  undertaken  is  nonincreasing  with  respect 
to  S.  They  have  also  been  used  to  measure  consumer  satisfaction  [8], 

In  the  maximum  weight  forest  problem  [see  (1.5)]  it  is  sometimes  more  real¬ 
istic  to  assume  that  the  objective  function  is  submodular  rather  than  just 
additive.  Three  other  examples  from  the  mathematical  programming  literature 
can  be  found  in  [11].  An  example  which  may  be  little  known  occurs  in  network 
flow  theory.  Given  a  network  with  edge  capacities,  a  source  s  and  a  set  N 
of  sinks,  let  Z(S)  be  the  maximum  flow  from  s  to  a  subset  S  of  sinks. 
Obviously  the  set  function  Z  is  nondecreas ing  and  Z(p)  =  0.  Fulkerson 
liked  to  ask  whether  Z  is  submodular  ia  his  course  on  network  flows.  It  is 
left  here  as  an  exercise. 


The  total  curvature  of  a  nondecreasing  submodular  set  function  is 


a  =  max  ^ 
jcN*  1 


p;(«)  -  P • (N  -  {j}) 


pj(0) 


where  N*  =  (j  €  N  :  P j (0)  >  0}.  Note  that  a  can  vary  between  0  and  1  and 
that  a  =  0  if  and  only  if  Z  is  additive.  In  Section  2  we  prove  that  the 
greedy  algorithm  finds  a  solution  with  a  value  which  is  guaranteed  to  be 
at  least  1/(1  +  a)  times  the  optimum  value  for  problem  (1.1)  when  X  is  a 
matroid  and  Z  is  a  nondecreasing  submodular  set  function  with  Z(0)  =  0 
and  total  curvature  a .  This  bound  generalizes  the  P.ado-Edmonds  theorem, 
see  (1.5),  as  well  as  the  bound  (1.7),  obtained  when  a  =  0  and  a  —  1 
respectively.  We  show  also  that  the  bound  1/(1  +  a)  is  best  possible  in 
terms  of  a. 


oiK 

Let  S  =  0  C  S  C  ...  c  S  be  the  sets  vhich  are  suceesively  con- 

K 

structed  in  the  course  of  the  greedy  algorithm  (S  is  the  greedy  solution). 
We  define  the  greedy  curvature  of  Z  as 

r  P •  (0)  -  P^S1) 

a  =  max  max  1  — - - — - - 

G  0£i<K-l  jeNi  PjCc1) 

where  N*  =  N*  n  {  j  £  N  -  S*  :  S*  U  { 3)  £  X }.  Note  that  <  a,  the  total 

curvature  of  Z.  Note  also  that  can  equal  0  even  when  Z  is  not  addi¬ 
tive.  In  Section  3  we  prove  that  the  greedy  algorithm  finds  a  solution 
with  a  value  which  is  guaranteed  to  be  at  least  (1  -a  )  times  the  optimum 
value  of  problem  (1.1),  again  with  the  assumptions  that  X  is  a  matroid 
and  that  Z  is  nondecreasing,  submodular  and  Z(0)  =  0.  Note  that  when 
aQ  ~  0  we  can  guarantee  the  optimality  of  the  greedy  algorithm  even  though 
the  objective  function  tray  not  be  additive. 

In  Section  4  we  give  a  bound  which  depends  only  on  the  matroid  X.  Let 

K  be  the  rank  of  X,  ie.  the  common  cardinality  of  the  maximal  independent  sets 

and  let  (h  +  1)  be  its  girth,  ie.  the  cardinality  of  a  smallest  dependent  set. 

We  prove  the  following  tight  bound.  The  value  of  a  greedy  solution  is  at  least 

l  2h-K 

half  the  optimum  value  if  K  >  2h  and  at  least  [ 1  -  —  (-£— )  ]  times  the  op¬ 

timum  value  if  K  <2h.  Our  bounding  method  is  based  on  the  weak  duality  theorem 
of  linear  programming  in  the  same  spirit  as  [1,11].  More  precisely  we  decom¬ 
pose  the  greedy  solution  Zg  ■  p  j+  P2+--.+  p^  where  p^  >  0.  Then  we  find  in¬ 
equalities  relating  the  optimum  value  Z*  to  this  decomposition  A0  >  Z*  where  p 
is  the  column  vector  of  p  ^  s  and  A  is  a  matrix.  Now  find  rr  >  0  such  that 

K 

rr  A  <  e  where  e  is  a  row  vector  with  K  ones.  We  have  Z_  =  £  j,  >„  A  p> 

G  i-1  1 

m 

(l  Wj)  z*  providing  a  bound  on  the  greedy  solution.  To  show  that  the  bound 
£s^tight  we  give  a  family  of  examples  which  achieve  it.  The  originality  of 
our  system  A  0  >  Z*  is  that  it  incorporates  simultaneously  information  on  the 
objective  function  Z  and  on  the  matroidal  structure  of  the  feasible  set. 
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Examples  of  independence  systems  are  quite  cormon  in  0,1  programming. 
In  fact,  given  a  nonnegative  matrix  A,  the  family  of  0,1  vectors  that 
satisfy  Ax  <  b  is  an  independence  system  (here  we  identify  a  set  S  and 
its  incidence  vector  x^  =  1  if  j  €  S,  0  otherwise).  Conversely  any 
independence  system  is  the  solution  set  of  such  a  0,1  program. 


Two  bounds  were  proven  in  [5]  regarding  the  greedy  algorithm  for  prob¬ 
lem  (1.1)  when  X  is  an  independence  system  and  Z  is  a  nondecreasing  sub- 
modular  set  function  with  Z(0)  -=0.  First  it  was  shown  that  the  greedy 

algorithm  guarantees  a  solution  value  at  least  [l  -  times  the 

optimum  value,  where  K  and  k  are  respectively  the  maximum  and  minimum  cardinali¬ 
ties  of  a  maximal  independent  set  in  X.  The  second  bound  is  -—r  where  p  is  the 

P  ' 

minimum  number  of  matroids  that  one  needs  to  intersect  in  order  to  obtain  the 

independence  system  X.  (The  fact  that  any  independence  system  can  be  expressed 

as  the  intersection  of  matroids  is  proved  in  [7].)  When  the  set  function  Z  is 

tc  1 

additive  these  two  bounds  can  be  sharpened  to  “  and  -  respectively  ( 6 ) ,  [7]. 

»  P 


In  Section  5  we  show  that,  in  terms  of  K ,  k  and  the  total  curvature 
of  Z,  the  greedy  algorithm  guarantees  a  solution  value  at  least  equal  to 

^  [l  -  ( j  ti™68  the  optimum  value.  This  bound  is  tight  for  all 
0  <  a  <  1.  Note  that  when  we  set  a  =  1  and  a  -*  0  we  get  the  bounds 

£l  “  ( ~K  ]  an<*  ^  respectively.  Note  also  that,  when  k  =  K,  we  get 


a  result  for  the  uniform  matroid,  namely  the  bound  (1  -  e  a)/a  ;  further¬ 
more  this  bound  is  best  possible  in  terms  of  a.  In  particular  it  is  easy 
to  see  that  it  dominates  the  bound  1/(1  + a)  for  any  0  <  a  <  1  since 
(1  -  e~a) / a  >  1  -  |  >  1/(1  +  ct)  . 


In  Section  6  we  give  a  bound  in  terms  of  p  and  a,  namely  the  bound 
l/(p+a).  This  generalizes  the  bounds  — — r  ,  —  and  mentioned 

earlier  in  this  introduction  for  different  variations  of  problem  (1.1). 

In  fact  the  result  is  proved  in  the  more  general  context  max  (Z(v)  :  v  E  X) 
where  Z  is  a  nondecreasing  submodular  vector  function  and  X  is  the 
intersection  of  p  polymatroids . 


. 
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2.  THE  SOUND  1/(1  +a) 


Let  N  be  a  finite  set  and  Z  :  2  -*  H  a  nondecreasing  submodular 

set  function  with  Z(0)  —  0.  Given  a  set  ft  c  N  and  an  ordered  set 
S  =  £  N,  we  define  S1  =  for  1  <  i  <  t,  and 


max 


p  (S1-1)  -  p.  (s1"' 

1  •  J  • 

J1  1 


u  ft) 


P,  (Si_1) 

Ji 


where  S*  =  {j.  £  S-ft  :  p.  (S1_1)  >  0).  Note  that  ao  <  a,  the  tonal 
1  i-1 

curvature  of  Z.  Denote  —  P-  (S  ),  i  =  l,...,t. 


Lzk“L4  2,1.  Z(ft)  S  p.  +  £  p.  +  X  p  (S). 

i:  1  i :  j^eftnS  1  wcft-S  w 

Proof  :  A  simple  consequence  of  the  definition  (1.4)  is 

(2.1)  Z(ft  U  S)  <  Z (S)  +  £  p  (S). 

ojeft-S  ^ 

By  the  definition  of  a 

0 

Z(ft  U  S)  =  Z(ft)  +  I  p.  (ft  us1-1)  >  Z(ft)  +  (I  -a  )  £  C. 

i:j.cS-ft  Ji  0  i :  j  . S -ft  1 

i  l 

□ 


In  this  section  and  the  next  two  we  assume  that  X  is  a  ratroid.  We 
K  /  / 

also  assume  that  S  =  is  the  sequence  chosen  by  the  greedy 

algorithm.  Note  that  is  a  lese  (i.e.  a  maximal  set  in  X) .  A  conse¬ 

quence  of  axiom  (1.3)  is  that  all  the  bases  of  X  have  the  same  cardinality. 
Recall  the  notation  S1  =  {jJ,...,ji}  and  p£  =  p  (S1_1),  i  =  1 . K. 

Ji 
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LEI-24A  2.2.  The  elements  of  any  base  S1K  =  {u> . .to,,}  can  be  ordered  so 

tnat  Pq)^(S  i  =  !»•••> K.  Fux*t h k.y-:ncT if  ok  €  n  then 

0).  =  j  .  . 

Proof  •  The  lemma  is  proved  by  induction  on  i,  for  i  =  K,...,1.  Assume 

that  the  elements  w.  satisfy  the  inequality  pu  (S^_1)  <  p.  for  £  >  i, 

i  K  ^  i 

and  let  Si  —  Si  -  {to^  :  £  >  i}.  Consider  the  sets  S1  and  SI1.  Bv  the 

matroid  axiom  (1.3), 3  uk  €  SI1  -  S1_1  such  that  SL_1  U  (un)  G  X.  Since 

j.  is  the  element  chosen  by  the  greedy  algorithm,  p  (S1-1)  <  p.  (S*-1). 

^i  J  i 

Furthermore  if  j .  €  ft1  we  can  set  ts.  =  i.. 

1  1  1  □ 

Q 

Let  Z  be  the  value  of  a  greedy  solution  and  Z *  the  optimal  value 
of  problem  ( 1 .  1 )  . 


THEOREM  2.3.  If  X  is  a  matroid  and  Z  is  a  Y.ond&cr  easing  subr.odular  set 
function  with  Z(0)  =  0  and  total  curvature  a,  then 


ZG  > 


1  +a 


Proof  :  Let  SI  be  an  optimal  solution  and  S  the  greedy  solution.  By 
Lemma  2 . 1 


Z*  «  a  z°  +  I  p.  +  2 

°  i:j.eS*nSK  1  i:W.eQK-SK  “i 

l 

By  Lemma  2.2,  p^  (SK)  <  (S*  <  p..  Therefore, 

l  i  1 


p,.  (sK). 


I*  <  a0  ZG  +  ZG  <  (1  +a)Z 


C 


□ 


G  1a 

COROLLARY  2.4.  The  proof  actually  s'ruows  the  strcyiger  bound  Z  >  y -  —  Z  . 
When  ao  =  0.,  the  greedy  algorithm  finds  an  optimal  solution. 


J.  ».  ^  n 


COROLLARY  2,5.  When  Z  is  additive  (ecu  ivale>itly  when  a  =  0)t 
algorithm  finds  an  optimal  solution  ( Fade -Edmonds  Theorem) . 


COEOILARY  2.6.  Z°>Z*/2.  (See  151.) 


CCROLLARY  2.7. 
lave  cardinal  it 


Any  two  maximal  sots  in  the  intersection  of  two  ma 
ie s  which  are  within  a  factor  of  2  of  each  ether. 


ro  ids 


FEW ARK  2.S.  It  is  worth  stressing  the  conbinator i al  spirit  of  the  deriva¬ 
tion  of  the  bound  1/(1  +  a).  This  derivation  is  based  on  two  observations, 

namely  Lemma  2.2.  and  the  inequality  (2.1).  It  is  close  to  the  classical 
proof  of  the  Rado-Edmonds  Theorem,  where  the  additive  version  of  Lemma  2.2 
is  used  implicitly. 


RE  (LARK  2.9.  The  proof  of  Theorem  2.3  can  be  modified  to  yield  a  stronger 


K  K  K  K—  I 

bound  :  instead  of  ft  and  S  ,  consider  ft  and  S  .  Then,  by  Lemma  2.1 


Z*  <  a(ZG -p.,)  +  I  p.  +  I  p  (SK~ S 

°  k  i:j.eftKnSK-1  1  i  V 


By  Lemma  2.2,  p  (S  )  <  p  (S  )  ^  p.  for  all  i  .  Therefore 

UJ.  W.  1 

1  1 


2*  <  (1  +qo)ZG  -  afl  p^  .  This  proves  the  bound 


(2.2)  T;^Pr 


CiRoiLARY  2.10.  If 
strict  in,:  r.,al  ity)  . 


Z*  #  o 


ana  a. 


#0  or  1 , 


non 


ZG  >  * 

1  +  a. 


L 


?roof  :  Assume  that  the  inequality  is  not  strict;  then  p  =0  as  a  con- 

K 

sequence  of  (2.2).  Then  in  every  base  there  exists  an  element  w  such  that 
K—  1  K 

Py  (S  )  -  0.  Therefore  p  (0)  =  0,  since  a  <  1.  The  greedy  and  optimal 
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solution  values  are  not  cfcarvcd  if  we  intersect  the  natroid  X  bv  the 

uniform  natroid  X^'  =  {T  :  j T }  <K-l}.  The  bound  (2.2)  becomes 

i  a 

*  7  ♦  ,  0  .  •  .  >  .  _ 

T+"oT"  L  +  T+  aT  *  Aram  by  our  assumption  we  must  nave  Dy.  j  =  0 

o  o 

and,  by  induction,  p.  =  0  Vi  =  1,...,K.  This  would  imply  Z*  -  0  ,  a 
cont  radiction. 


□ 


CidOLZAPY  2.11.  If  Z*  #  0  and  aM  or  1.  thon  Z  >  Z*  (a  svriot 

-  1  +  a 

C  'f\  tr  ( 1  i  7  v)  . 


Next  we  show  that  the  bound  1/(1  + a)  is  best  possible  in  terms  of  a. 

In  turn  this  implies  that  the  bound  1/(1  +  o<0)  is  best  possible  in  terns 

of  a  . 
o 


.  niei.EM  2.12.  There  exists  an  infinite  family  of  prebt  or.s  such  that 
,G  . 


K 


1  7* 

TT£  zk  as  K 


-»  <» 


VC.t'e  ZJ 
the  Ktn  : 


and  Zj^  t2  0  are  rer't  natively  the  greedy  and  a  pi  l-al  values  of 


r: ;  err.. 


Proof  :  LTten  a  -  0  the  bound  is  always  tight,  so  there  is  nothing  to 
prove.  When  a  =  1,  the  result  is  already  known  [5).  So  assume  0  <  a  <  1. 


Let  N  =  { j 
for  t  =  1, . . . ,K. 
such  that 


• • • . iK,  w, 

Ke  define 


. . . ,uK)  and  Nt  =  { j  i , 
X  as  the  family  of  all 


•  •  >  3 1  *  'J  j  ,  •  •  .  , Wj. } 

the  subsets  S  c  N 


(2.3)  |s  n  Nc|  <  t  for  t  =  J,...,K. 


It  is  clear  that  X  is  an  independence  system,  i.e.  axiom  (1.2)  is  verified. 
So  to  prove  that  X  is  a  matroid  it  remains  to  show  that  axiom  (1.3)  holds. 
Let  S ,  T  £  X  be  such  that  j  T  |  +  1  ~  !  S  j .  Then  S-T^S.Let  £  S  -  T 
be  an  element  with  largest  index  i,  1  <  i  <  K,  where  denotes  either  j ^ 

or  a>..  We  will  show  that  T  U  { e . }  £  X,  namely  that  I  (T  u  {e .  })  O  j  t 

1  1  ,  t  1  t 

for  t  =  1,...,K.  By  the  choice  of  e^,  jS  n  N  |  >  ]  (T  u  {e^})  n  N  j  for 

t  >  i .  This  implies  |(Tu{e.})nNcj<t  for  t  >  i ,  using  the  fact  that 

^  t  t 

S  is  independent  and  (2.3).  When  t  <  i,  (Tu  {e^})  O  N  =  T  O  N  ,  so  the 

inequality  j  (T  u  { e  ^ } )  ON  j<  t  for  t  <  i  follows  from  the  fact  that 

T  is  independent.  This  shows  T  U  {e.}  £  X  as  announced. 

r  l 

Define  the  set  function  Z,  for  any  S  C  N,  as 
Z(S)  -  I  {o^  :  j.  £  S}  +  S  {p^  :  €  S  such  that  i=  1  or  £  S]  + 

— I  {p.  :  u.  £  S  such  that  i  >  2  and  j.  .  £  S),  where 
1  -Ci  x  i  1-1 

C.  =  — —  ,  for  i  -  1,...,K.  This  function  is  submodular,  non- 

i  1  +  a  ’ 

decreasing  with  total  curvature  a  since 


0  (S)  = 

;u . 

l 


- p.  if  i  >  2  and  j .  .  £  S 

I  -a  i  i-l 


if  i  =  1  or  ji_j  £  S, 


(S)  - 


if  u<.+]  £  S 


Pi  "  TTa  P.  ,  =  (1  “a)pi  if  wi  +  l  £  S‘ 
l+l 


Sow  we  compute  the  value  of  a  greedy  solution.  The  largest  discrete 

derivative  at  0  is  o.  (0)  =  P  (0)  =  (0)  =  t^ie  £r<::e^y 

J»  1  ?  t 

algorithm  can  choose  j,  in  the  first  iteration.  Assume  S  -  {  j  i .} 

has  been  chosen.  p.  (S^)  —  0  .  »  S  U  {uk}  £  X  for  i  ^  t  and 
Jt+1 

0  (Sc)  <0  .  for  i  >  t+  1.  So  i  ,  can  be  chosen  next.  The  greedy 

(jj.  t+1  ‘  t  +  i 
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K  K 

Z*  <  -  »„.<«  <  C,  •  z  Pi- 

1*1 


K 
I 

G'  i=2 


?G  P  j  ^ 

Therefore  — r  >  1  -  a  +  a_  — r .  Since  Z  <Kp  ,  the  validity  of  the 
Z  G  G  2  1 

hound  (3.1)  is  proved. 


The  fact  that  the  bound  can  be  achieved  is  shown  by  the  following  exam¬ 
ple.  The  ratroid  has  K+  1  elements  and  only  one  set  is  infeasible,  namely 
the  full  set.  The  set  function  is  defined  by 


Z(S)  = 


is| 

if  x]  i  S 

8  ♦  Js|  (1  -  8) 

if  x.  e  s 
* 

Tt  is  easy  to  check  that  Z  is  nondecreasing  and  submodel ar  for  0  <  8  <  ^ » 

z<=  K_, 

that  =  8  and  that  — y  =  1  -  ^ —  if  the  greedy  algorithm  chooses 

Z 

x j  in  the  first  iteration. 

□ 


r, K  The  bound  -  ct^  ^  can  easily  be  computed  in  the  course 

of  the  algorithm.  It  gives  an  a  posteriori  bound  on  the  quality  of  the 
greedy  solution  which  can  be  tighter  than  the  a  priori  bound  1/(1  +  ^i) . 

In  fact  it  proves  the  optimality  of  the  greedy  algorithm  when  =  0, 
which  occurs  when  Z  is  additive  but  may  also  occur  for  more  general  set 
functions. 


COROLLARY  3.3.  If  X  is  a  matroid,  Z  a  submodular  set  function  and  the  greedy 
algorithm  is  such  that  0j  (S*)  *  Oj(0)  ^or  *  “  !■»••• »  K-l  and  *11  j  €  N*,  thei 
the  greedy  solution  is  optimum. 
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r.z’-1-.RK  c.4.  Theorem  3.1  remains  true  if  Ct^,  is  replaced  by  the  parameter 


aG  = 


max 

jO:-SK_1:p.  (p)>0 


c.(0)  -  Pj(SK-1) 

c.(0) 


or 


jfN-  (j,) 


J 

Pj(0)  -  P j (N  -  {j}) 


Note  that  the  parameters  Cl  and  Q  do  not  dominate  each  other.  So  the 

G  G 

two  versions  of  Theorem  3.1  are  interesting.  It  is  also  worth  noting  that 

p  (0)  -  p  (N-{j  }) 


a  =  max 


••  l 


‘G  ’ 


Pt  (0) 
J  1 


However,  the  worst-case  example  of  Theorem  3.1  works  for  but  not  for  a. 


4.  A  BOUND  IN  TERMS  OF  THE  RANK  AND  THE  GIRTH  OF  THE  MATROID 


q  Ik 

Let  S  =(*>,  S  . S  be  the  sequence  of  sets  chosen  by  the  greedy  algorithm, 

and  define  0j  *  z (s ^ )  -  Z(S^  *) .  Note  that  the  greedy  solution  has  the  value 
2  =  Oj+  02+-«-+  oK- 

It  has  been  shown  in  [11]  that  Z  is  submodular  and  nondecreasing  if  and  only 
if  Z(q)  <  Z  (S)  +  j  q  (S)  for  all  q,  S  £  N.  let  q  be  an  optimal  solution. 

J€n-s  J 


Z*  =  Z(Q)  <  Z(SC  )  +  S  p- (St),  0  <  t  *5  K. 

jeers  J 


(4.  1) 


Tor-  all  t  <h  and  j  £  p  -  S*  ,  SC  U  [  j}  €  X,  and  therefore  p(St)<o.  +  j. 

t  ^  1 

Since  j  ^  -  S  |  <  K,  we  obtain  that  Z  must  satisfy  the  following  relation¬ 


ship: 


Z*  <  I  p.  ♦  Kp  ,  t  =  D . h  -1. 

i  =  I 


(4.2) 
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For  independence  systems,  it  turns  out  that  these  constraints  are  the  only 
essential  ones  in  the  analysis  of  the  greedy  heuristic,  see  [53.  For  a 
matroid,  however,  the  optimal  solution  must  satisfy  another  fanily  of  in- 
<  quali ties. 


PROPOSITION  4.1.  The  elements  of  any  basis  of  a  matroid  can  be  ordered 
so  that,  for  any  h  <  t  <  K, 


p  (S < 

(«i 


Pk  if  1 «  h 

P  j  if  h  <  i  <  1 

Pt+1  if  i>t. 


Proof.  Consider  the  order  defined  in  the  proof  of  Lemma  2.2.  Since 
S*  1  U  {io^}  is  independent,  so  is  S  U  {»^}  for  every  t  <  i.  The  ref 
by  the  choice  rade  in  the  greedy  algorithm  (S  )  <  Pt  +  ]  ^or  i  ^  £  ■ 


For  h  <i  <r.  0  (S1”*)  <  p.  by  Lemma  2.2;  this  implies 

tfij  1 

t  ^1  t 

P  (S  )  <  p.  since  S  C  S  . 

(I>£  1 


Final  lv,  for  i  <  h,  p  (Sh_1)  <  p. 

1 


:  l  nee 


oh-1 


U  )  is  i ndf pendent . 


Therefore  c  (SC  )  <  p.  ,  as  a  consequence  of  the  hypothesis  h  <  t. 


□ 


Frcposition  4.1  allows  us  to  write  the  inequality 


S  t  p  (S  )  <  h  ph  +  Pfcf  + 
JcCl-S  J 


+  Pt  +  (K  -  t)pt+] ,  h  <  t  <  K  -  1  (4.3) 


Cv-bining  (he  i t. qual i t ies  4.1  and  4.3  we  get  : 

Z*  <  P,  4  •••  4  Ch-1  4  (h  +  >)Ph  4  2  Ph+,  +  ••  •  +  2ot  ♦  (K-t)p  (4.4) 


for  h  t  <  K  -  I.  We  have  just  proved  : 
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THEOREM  4.  2  .  The  following  t>.<  q- tali  ties  are  all  valid  : 


*'<z,  ci 

1  =  1 


0<t<h-l  '4.5) 


Z*  <  S  p.  +  Ch  +  1)  p.  +  S  2  p.  +  (K  -  t)  p  h<t<K  -  1 
i  =  1  1  .  n  i  -h  +  1 


(4.6) 


0  <  p  -  p 

wt+l 


0<  t  <  K  -  1  (4.7) 


The  inequality  4.6  for  t  =  K  -  1  is  always  dominated  by  the  one  correspond¬ 
ing  to  t  =  K-2  and  will  be  removed  from  the  system. 

Now  we  use  the  bounding  technique  presented  in  the  introduction.  Thus, 

any  n  >  0  which  is  a  solution  of  the  following  system  yields  a  bound 
K- ! 

E  it.  for  the  performance  of  the  greedy  algorithm, 
r =  1  1  I 


h  +  1  K-h 
h  +  1  2  K  -  h  -  1 


h+  1  2 


.  2  2 


1  -  1 


1  -  1 


hth  row 


0  (K  -  1)  row 
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We  decc-pese  T,  into  two  vectors  T.  =  (u,v),  where  u  -  (u^  :  i  —  -1) 

is  associated  with  the  first  (K  -  1)  rows  of  the  above  matrix  and 
v  -  (v^  :  i  K-l)  is  associated  with  the  regaining  rows. 


(4.8) 

2  <  t<h  -  1  (4.9) 


K-l 

K  Uj  + 

I  u.  +  v. 

i-2  1  ’ 

<  1 

K-l 

K  + 

—  u.  +  v 
i  i  t 

i=t  + 

-  Vt-1  <  1 

K-l 

K  a.  +  2  (h  +  l)u.  +  v.  -  vl  ,  <  1 

h  .  u . ,  x  11  “i 

i=n+l 

K-l 


(4.10) 


(K  -  t  +  1  )u  +  2  2u.  +v  -v  .  <  1,  h+l<t<K-l  (4.11) 

C  i=t+l  1 

To  compute  an  analytic  solution  of  this  system,  we  consider  two  separate 
cases  :  K  >  2h  and  K  <  2  h. 

When  K  ^  2h  ,  setting  u^  =  0  for  1  <  i  <  K  -  2,  ujj_j  =  v.  =  j 


f  or 


1  <  i  <  h  -  1  and  =  0  for  h  <  i  <  K  -  1 ,  we  get  a  hound  of  value  ^ 


When  K  <  2  h ,  we  set  u .  =  0  for  2h-K+l<i<K-2  and  v.  —  0 

1  .  1 
for  1  <  i  <  2  h  -  K  and  h  <  i  ^  K  -  1 .  (Note  that ,  in  the  rases  ^  -  K  or 

K-  1,  the  whole  vector  v  is  set  equal  to  0  whertas  no  u^  is.  In  th*se  two 

cases  the  remaining  system  is  t  r  i  angul  ar .  )  When  h  <K-2,  the  regaining 

system  is 


Kut  +  ut  +  l  +--’  +  U2h-K  +  Vl 


<  1  1  <  t  <  2  h  -  K 


UK-1  <  1  -  Vt-1  '  Vt 


(h  +  1 ) Uj._ j  <  1  +  Vi 

2  Vi  <  1 


2  b  -  K  +  1  <  t  <h-i 


t  -  h 


h  +  1  <  t  <  K  -  1 


(4. 12. t) 
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U’fc  consider  the  solution 


_  h  /k-  A 
ui  ~  \  k  / 


K  -  h 
UK- 1  "K* 


h  (K  -  2  h+  i) 
Vi  K 


2h-K-i 


1  <  i  <  2h  -  k 


(when  h  <  K  -  2) 


2  h  -  K  <  i  <h  -  1 


U’e  now  prove  that  this  solution  is  feasible.  KTien  h<K-2,  the  in¬ 
equalities  (4.12.t)  are  verified  for  t  >  h+1  as  a  consequence  of  the 
at  s  pt  i  on  K  <  2  h.  For  2h-K  +  l<t<h,  t  .e  i  n«  qual  i  t  5  es  are  sa  t  i  s  f  i  ed 
with  equality,  i.e.  when  t  =  h 

*♦»<*-'.  -  -•  *  ’  '  *  v, 

and  when  2h-K  +  1  <  t  <  h  -  1 


K  -  h  .  h  . 

Vr  "r  =  1  '  ii  =  1  '  vt-i  ’  vt 


The  values  u.,  1  <  i  <  2h-K,  are  obtained  by  solving  at  equality  the  tri¬ 
angular  system  (4  12. t),  1  <  t  <  2  h-K  : 

Kut  4  Ut_,  4  ...  4  U2h_K  =  |  (  -  1  -  Vl). 

(The  cases  h  =  K  or  K-  1  are  also  obtained  by  solving  the  correspond! ng 
triangular  systems  at  equality.) 

Since  (4.8)  holds  with  equality,  the  value  of  the  bound  is  : 


I  u.  -  !  -  v,  ~  (K-  Du,  =  1  -  \  (Kr‘) 
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Note  that  the  l  est  hound  which  can  be  obtained  from  the  system  (4.S)  - 
(4.11)  is  the  optiral  value  of  the  linear  program 

K-l 

rax  E  u .  ( 4. 1 3) 

i=  1  1 

subject  to  (4.8)  -  (4.11) 
and  u  >  0,  v  >  0. 

VCe  claim  that  the  solution  derived  in  this  section  is  indeed  an  optiral 
solution  of  (4.13).  To  check  it,  it  suffices  to  exhibit  a  feasible  solution 
of  the  dual  linear  program  with  the  same  objective  value  : 

K 

rin  I  0.  (4-lM 

i-1  1 

subject  to  (4.5)  -  (4.7)  vith  1 *  set  equal  to  1 
and  p  h  0. 


I'e  propose  the  following  solutions  : 

h;u-n  K  ^  2h  ,  take  p.  =  for  1  <  i  <h  and  p^  -  0  Tor  h+  1  <  i  <  K. 
When  K  <  ?h,  take 


(k  -  n1-1 

p.  =  for  1  i  <  2  h  -  K 

K1 


(K-l)  ^ 

=  2h  for  2h-K+l<i<h 


(4.15) 


Pi  “ 


0 


for  h  +  1  <  i  <  k. 
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The  interested  reader  can  verify  for  himself  the 

K  1 

solutions.  The  fact  that  pj  =  when  K  >  2  h 

2h-K  /'K-l\2h"K 

K  <  2h,  *•  P-  —  1  -  (  — j. —  )  (geometric  series) 


I  K  o  =  K-h/K-lVB 

i=2h-K+l  Pi  K  V  K  / 


2h-K 


Therefore 


-  Pi  =  1 
i=l  1 


-  *  /K  -J_\ 
K  {  K  ) 


2  h-K 


feasibility 
is  obvious. 

and 


of  these 
When 


(4. 16) 


Now  we  show  that  the  bounds 


Z 


G 


Z 


* 


1  - 


I  (Vf  ‘ 


K  >  2h 

K  <  2h 


obtained  above  are  tight;  that  Is,  we  exhibit  families  of  matroids  and  sub- 
modular  nondecreasing  set  functions  for  which  the  greedy  performance  satisfies 
the  above  bounds  with  equality.  We  define  a  matroid  on  the  set  of  elements  de¬ 
noted  by  B  \  J  A  i  i  T .  where  | B|  *  h,  )Aj  -  h  |T|  *  K  -  h.  The  elements  in  B  will 
be  the  first  elements  chosen  by  the  greedy  algorithm,  the  elements  in  A. will  belong 
only  to  the  optimal  solution  and  the  elements  in  T  will  be  common  to  the  greedy 
and  the  optimal  solution  Let's  define  an  independence  system  in  the  following 
way.  The  independent  sets  are  all  the  sets  of  size  at  most  K  not  containing 
more  than  h  elements  in  the  set  B  (J  A.  The  sets  of  h  elements  in  B  M  A  are 
called  critical  sets. 


PROPOSITION  4.3  The  independence  eastern  is  a  r.airoid. 

Pr oof.  Tt  is  the  direct  sura  of  two  uniform  r.atroids,  [12).  It  is  also  easy 
to  check  the  r.atroid  axioms  (1.2)  and  (1.3). 


□ 
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Ve  now  examine  the  case  K  >  2h  and  define  a  nondecreasing  suhmodular 
set  function  Z  which  gives  the  worst  case  of  j .  The  subsets  of  A  of  a 
given  cardinality  will  be  indistinguishable,  as  far  as  the  value  of  Z  is 
concerned.  So  we  will  denote  by  any  subset  of  cardinality  j.  Similarly 


the  subsets  of 

Bu  T 

of  a  given  cardinality  will  be 

we  will  denote 

w1  C  B 

U  T  any  subset  of  cardinality 

Z(0) 

-  0 

Z(K1 

U  Aj)  * 

L±A  ; 

2  h 

zew1 

U  Aj)  = 

1  +  JL  i  >  h. 

2  2h 

PROPOSITION  UA_  The  function  Z  is  eul^dular  and  no.dec^a^ng . 

The  proof  is  very  easy.  It  is  left  to  the  reader 


The  set  B  can  be  chosen  first  by  the  greedy  algorithm  because  the 
increment  given  by  any  element  x  6  B  is  not  sraller  than  the  increment 
given  by  any  other  element,  when  the  set  of  elements  has  cardinality  less 
than  or  eoual  to  h  (ties  are  broken  arbitrarily).  At  stage  h,  the  only 
elements  which  give  a  positive  increment  are  the  elements  a  6  A,  but  they 
form  circuits  with  the  set  B,  ’(cause  it  is  a  critical  set;  thuiefore  : 

Z°  -  Z  (B  u  T)  •-  j  . 

Since  |T|  =  K  -  h  >  h ,  Z(TUA)  =  |  ♦  1  s  1. 


For  the  case  K  <  2h,  we  use  the  matroid  defined  earlier.  However,  we 
need  to  partition  the  set  B  in  two  subsets,  one  will  still  he  denoted  by  B> 
the  other  by  Y.  Namely,  let  B  be  the  set  of  the  first  2h  -K  cl.r.nts 
in  the  greedy  solution,  Y  the  set  of  the  next  K  -  h  flci-rts  in  the  greedy 
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solution,  T  the  set  of  K-h  elements  common  to  the  greedy  end  the  optimal 
set,  and  A  the  set  of  h  elements  only  in  the  optimal  solution.  We  denote 
by  B  C  B,  W"*  C  Y  U  T  and  A*"  C  A  subsets  of  cardinality  i,  j  and  m  , 
respectively.  We  consider  the  following  function  : 


• 

l+r 

ZiB1  u  WJ  u  A”)  = 

P‘  * 

(q+m)pi  +  r+l 

i  +  j.  <  h 

2h-K 

Z(B 1  U  yj  U  Am)  = 

r  p  + 

t=i 

(K-h  *m)r2h  K+1 

i  +  j  >h 

where  q  -  min  [j,  K-h] 

r  =  max  [0,  j  -  (K  -h  )  ] 
and  pt>  1  <  t  <  K,  is  given  in  (^.55). 

Note  that  the  function  is  doubly  defined  when  i  +  j  -  h.  It  is  easy  to 
verify  that  the  two  expressions  are  then  identical  since  j  >  K-  h  (a  con¬ 
sequence  of  the  fact  that  i  <  2h  -  K) .  For  the  proof  of  our  next  theorem, 
we  will  find  it  useful  to  have  both  expressions  available. 

'J  !i  HO  REM  4.5 .  Trie  function  Z  defined  above  is  sub-'odulac-  and  ncr.ieci-essi 

The  proof  is  strai ghtforward,  though  somewhat  long.  Anyone  interested  can 
find  the  proof  in  the  appendix. 


Again,  the  set  B  can  be  chosen  first  by  Greedy,  because  when  i  <  h 
B^  c  B  and  £  €  B  -  B*,  we  have  p^(B^)  =  p.  +  j,  v’bich  is  equal  to  the 
increrent  given  by  elements  in  Y  U  T  or  in  A.  When  all  the  elements  in 
have  been  added  to  the  greedy  solution,  we  have  i  =  2h  -  K  and  the  elirrents 

in  Y  give  increments  p  „  .  because  r  is  equal  to  zero.  After  that, 

2h  _  * 

elertnts  in  A  give  a  positive  ini  lament,  but  3  IM’  is  a  critical  set  and 
the  addition  of  an  element  in  A  would  create  a  circuit.  Therefore,  only 
elements  in  T  can  be  added,  but  since  i  + j  -h,  they  give  null  increments. 
Therefore,  we  obtain  : 


Z  "  Pi  * 


P2h-K  +  (K  “  h)f^h  -K+  1 


If  we  consider  the  solution  T  U  A,  the  eh  rents  in  T  give  i  in  1 . r . nt s  of  p,, 
es  well  ns  the  el. rent  in  A.  Therefore, 


,*  (K-h)  ,  h 

Z  _  +  *  =  1. 


And,  by  16, 


—  =  1 
L 


5.  A  TIGHT  BOUND  FOR  INDEPENDENCE  SYSTEMS 


In  this  section  we  consider  instances  of  problem  (1.1)  where  X  is  an 
independence  system.  As  earlier  we  assume  that  Z  is  a  nondecreasing  sub- 
modular  set  function  with  Z  ($)  =0.  Let  =  0  and  SC  =  f • 
t  =  1,...,  i ,  be  the  successive  sets  chosen  by  the  greedy  algorithm.  Note 
that  k  <  1  <  K,  where  K  and  k  are  respectively  the  maximum  and  minimum 
cardinality  of  a  maximal  set  in  X  (K  and  k  are  sometimes  called  respect¬ 


ively  the  upper  and  lower  ranks  of  X.)  Recall  that  B.  *  p  . 

1  Ji 


(Si_1)  and 


that  y  denotes  the  total  curvature  of  Z.  (In  this  section  y  could  be 
replaced  by  y.  defined  as  in  Section  2  with  q  being  an  optimal  solution  and 

l)  k 

and  S  being  the  set  S  .) 


5.1  u>iy  vident  set  Q  u  nd  t  =  0,...,k-l, 

Z(fi)  <  a  -  p  +  S  p.  +  (K-s)p  . 
irj.-St-n  iij.eHnSt 

J  l  J  l 

s  =  (fi  n  s C  j  . 

Proof  :  Follows  from  Lemma  2.1  and  the  observation  that  S*1  U  {u}  is  in¬ 
dependent  as  a  consequence  of  the  assu-ption  t  <k-l. 


13 
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We  will  only  use  Lemma  5.1  when  q  is  an  optimal  solution. 

Consider  the  family  f  of  all  instances  of  problem  (1.1)  where  X 

K  ,  K  ,  Q 

and  Z  have  the  given  parameters  K,  k  and  a  as  defined  above.  For  sim¬ 
plicity  of  notation  we  write  F  =  F  ,  .  For  0  <  s  <  k,  let 

K  ,  K  ,  Cl 

l<ij<...<i  <k  be  a  sequence  of  integers  and  let  F(ij,...,is)  c  F 
be  the  family  of  problems  such  that  a  greedy  solution  has 


the  elements  j. 


3: 


in  common  with  an  optimal  solution 


£' 

Note 


*i  -s 

that  when  s  =  0  the  set  of  common  elements  is  empty. 


Q  *  *  # 

Let  Z  and  Z  be  the  values  of  a  greedy  and  optimal  solution  respect¬ 
ively.  As  a  consequence  of  Lemma  5.1,  for  any  problem  in  F(i ^ ,  .  .  .  ,  i  )  , 
Z^>B(i  ,...,i  )  Z*  where 


(5.1) 

an<J 


B(i; . i  )  =  Min  I  p. 

b  i  =  l 


subject  to  >  0,  i  -  l,...,k 


K 

* 

a 


K 

a 

a 

a 


K 

1  K-l 

1  a 


a  1  a 


K-s  +  1 
1 


K-s 

a 


a  •  •  •  a  K-s 


^  i 
o. 


- - - -  -.--v 
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5--  B(i 


,  i  )  >  B (c)  C'.tsper  i  ^s^-s-.oe  !  <  i  <  —  <  ig^k. 


Proof  :  Assure  s  >  1  and  consider  i  for  1  <  r  <  s.  For  simplicity  of 

notation  we  denote  c  =  i  .  First  we  show  that  d  ^  f  ,  in  sere  optiral 

r  q  q+1 

solution  of  the  linear  program  (5.1)  associated  with  B(i ^  , . . . , i  > .  Assume 
not,  i.e.,  assume  that  pq  >pq+j.  The  inequalities  q  and  q+1  of  the  system 


a  p  +  ...  :  ao  ,  x  (K  -  r  +  l)o  >1 
i  q~l  q 


aPj  +  ...  +  apq_j  +  pq  *  (K  -r)Dq+,  >  1. 


Note  that  the  first  of  these  two  constraints  is  not  tight.  Decrease  the  value 
of  pq  by  C  >0  small  enough  so  that  the  inequality  rerains  feasible,  and 

^da  j^_  r  to  p£  for  q  +  1  <  i  <  k.  It  is  clear  that  this  now  solution  is 
feasible.  The  objective  value  of  the  linear  program  is  modified  bv 
£• 

(k_q)  -  £  <  0  since  k  <  K  and  q  >  r.  Therefore  o  <p  ,  in  some 

q  q  +  1 

optiral  solution  of  (5.1). 

New  assume  q  =  i  <  ir+]  ~  1.  Denote  by  A  the  constraint  ratrix  of 

the  linear  program  (5.1)  associated  with  B(ij , . . . ,i^_ j , if,i j , . . . »ig) 

and  by  A1  the  constraint  matrix  associated  with  B(i  ,,i  +  1  , i  ,, 

...,ig).  A  and  A*  only  differ  by  their  colur.ns  q  and  q+1.  Thus  any 

vector  p  which  satisfies  1  <  A  p  and  p  <p  ,  also  satisfies  1  ^A*c. 

q  q+ 1 

This  implies 


B(i  ,...,i  ,,i  ,i  ,,...,i  )  >  B(i  —  1 , i  +  1,  i 

:  r-1  r  r+ i  s  i  ’  ’  r  r 


r+1 . s 


Popeating  iteratively  this  argument  for  all  1  ^  r  <  s  such  that 

i  <  i  ,  -  1 ,  we  obtain 
r  r+1  ’ 


B(iif...,i  )  >B(k-s  +  l,  k-s  +2,...,k). 


New  let  A£  be  the  constraint  matrix  associated  with  B(k  -s  +  l,k-s+2, 

...,k).  Anv  vector  p  which  satisfies  1  <  A  C  ?nd  p,  <  C  „  <  . . . 

s  k-s+1  k-sf2  ' 

^  p,  also  satisfies  1  sj  A  .  p.  This  shews 

K  S+  1 

B(k  -  s  +  1,  k  -  s  +  2,  .. .  ,k)  >  B(k  -  s  +  2,. .  .  ,k)  >  ...  >  B(k)  =  B(p)  . 


□ 


&2LL*-  *<«  >  i  [i  -  ] 


Froof  :  Consider  the  linear  program  (5.1)  associated  with  E(p), 
h  1  /K  -Q\k-t 

the  tth  constraint  by  -  ^  —  j  and  add  all  the  constraints. 


"u 1 t i ply 


t:,'l  <i:1  i lv  k  )  j  J  • 


Su~.ming  the  geometric  series  we  observe  that  the  coefficients  of  p.  equal  1 
for  every  i  =l,...,k  and  that  the  left-hand  side  of  the  inequality  equals 

a  [J  "  (nr)  ]  as  required- 


"rr'ZkZX  5.4.  If  X  is  an  independence  st-sier  L’iik  upper  rank  K,  leaser 
yank  k,  ana  if  Z  is  a  nondeor easing  sub-nodular  set  func-ii-cn  vzth  Z(j>)  =  0 
and  total  cur  suture  a,  then 

-■hi  this  l:und  is  tight  for  all  0  <  a  <  1  and  k  <  K. 

Proof  t  The  bound  is  valid  as  a  consequence  of  Tcr.-as  5.2  and  5.3  and  the 
fact  that  F  —  (~){F(ii>...,i^)  :  i  ^  i  ^  <...  <i^<k  is  a  (possibly 
empty)  integer  sequence}.  The  fact  that  the  bound  is  tight  is  shown  by  the 
following  worst-case  examples. 
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Let  N  =  {j1,j2,....jK_1.u,.-.*.«K>  and  let  x  be  the  faTT,il>'  of  a11 
the  subsets  S  C  N  which  contain  at  rost  k  elercnts  if  j  G  S  end  at  most 

K  if  jj  £  S,  where  k  <  K.  Define  p^  -=  ^  f°r  *  ~ 

and  consider  the  set  function  defined  on  the  subsets  of  S  as 


...  .  .  K  -  ui  v  ^  u 

Z<Ji.,3i.****,3iJ  K  h=i  Pjh  K‘ 


1  2 


t  1 


In  this  forpula  we  allow’  t  or  u  to  take  the  value  0.  The  s .mation  is 
taken  to  he  0  if  t  =  0.  Therefore  Z(0)  =  0.  Note  that 

P-  (ji  ,*••,.]£  ,  » •  ••  »“r  )  =  (1  -  a  ^  )  p.  >  (1  -a)p. 

Ji  i  •  t  J  u 

!  t  , 

Pw>  (jf  ♦•••’ii  »  “r  ••••*a)r  5  =  K  ~  K  .S.  pj.  >  (1  _Ct)  K  * 
r  i  t  l  u  h=l  Jh 

This  shows  that  the  set  function  Z  is  submodular,  nondecreasing  and  has 
total  curvature  a  . 


The  optimal  solution  of  problem  (1.1)  is  {to  ,  ...,u>  }  with  value  1  . 

1  .  1  .  K  .  .  . 
Since  p  =  -  ,  the  greedy  algorithm  can  choose  j  in  the  first  iteration, 
IK.  1  • 


*  X  1 

•e  it  has  chosen  S1  1  =  { j  j , . . . ,  j }.  Then  (S1  S  =  Pi  =£ 


Assume 


whereas  p 


i-1 


1 

1 

K  -  a\  i-1  ‘ 

_  1  / 

K  -aV 

K  ~ 

K 

1  ‘  l 

.  K  J  J 

“  K  \ 

a-l 


r  h=l 

So  the  greedy  algorithm  can  choose  the  element  j.  in  the  itb  iteration. 

k  •  r  /v  _„\k 

1 


The  greedy  solution  has  the  value  Z  p,  =  — 

..ha 

h=  1 


ffl] 


as  required. 


□ 


CCr.OL 


Li  ARY  5.5.  [5]  ZG  >  f  1  -  )  ]z*. 


Proof  :  Set  a  =  1  in  the  bound  of  Theorems  .4. 


□ 


CC  ~  Oil  ARY  5,6.  [6]  ,[71.  If  X  is  an  indeiendcnce  tv- star.  and  Z  is  additive. 


.  „G  .  k  _* 

tncn  Z  ^  ^  Z  i 
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Proof  :  Let  a  -»  0 

('  -  f)‘  -  '  -  f 


in  the  bound  of  Theorem  5.4.  The  fact  that 
implies  the  result. 


□ 


CCr.OLLARy  5.7.  If  X  is  a  unifcim  natroid  and  Z  has  total  curvature  a } 


:  hen 

zG  > 

j[ 

i r)>- 

CuROi  j 

IARY 

5.8. 

[2] 

,[11].  If  X  is  a  uniform  natroid ,  then 

zG> 

1  - 

(- 
\  K 

if] 

Z*. 

Proof 

:  Set 

a  =  1 

in  Corollary  5.7. 

CCr.Ol 

lap.y 

5.9. 

If 

X  is  a  uniform  matr-cid  and  Z  has  total  cu 

t hen  ZG  >  IrJL!!z*>(l-§)  Z*. 


Proof  :  For  any  integer  K,  <  e"01.  Therefore  the  bound  follows 

from  Corollary  5.7.  Furthermore  - — — —  >  \  -  j  for  ail  0  <a  <  1. 


6.  THE  BOUND  l/(p*a) 


The  last  result  that  we  shall  prove  concerning  problem  (I.  I)  is  the 
following.  Let  X  be  an  independence  system,  p  the  minimum  number  of 
r.atroids  that  one  needs  to  intersect  in  order  to  obtain  X  and  Z  a  non¬ 
decreasing  submodular  set  function  with  Z(jZ’)  =  0  and  total  curvature  a 
Then  the  greedy  algorithm  finds  a  solution  with  value  Z  >  p- Ta  ^  * 
where  Z*  is  the  optimal  value. 

t 
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Howcver  we  derive  the  bound - for  a  more  general  model  than  (1.1) 

p  +  a 

as  an  exanple  of  a  possible  extension  of  the  results  in  this  paper. 

Given  a  nonnegative  vector  u  —  (u.  :  j  €  N),  let  |u|  =  —  u.. 

N  "*  ^  K 

Given  two  vectors  u,  v  €  F  we  define  w  =  u  vv  as  the  vector  of  F  with 

components  w.  =  max  (u.,  v.)  for  all  j  £  N.  An  integral  poly'iotroid  is 

J  J .  J  fj 

a  pair  (N,P)  where  N  is  a  nonempty  finite  set  and  P  C  F+  is  a  finite 

family  of  integral  vectors  such  that 


(i)  v  £  P,  u  <  v  and  u  is  an  integral  vector  ■*  u  £  P,  and 

(ii)  u,  v  £  P  and  |u|  +  1  =  |v|  •*  3  w  £  P  such  that  u  <  w  <  u  vv. 


The  vectors  in  P  are  called  independent  Vectors.  The  concept  of 
integral  polyir.atroid  was  introduced  by  Edmonds  [4]  as  a  generalization  of 
matroids  (obtained  when  P  contains  only  0,1  vectors.  Then  the  indepen¬ 
dent  sets  of  the  matroid  are  precisely  the  subsets  of  N  whose  0,1  inci¬ 
dence  vectors  belong  to  P.)  An  introduction  to  integral  polyr.atroi ds  can 
be  found  in  [12].  A  known  property  is  that  P  can  be  written  as 


(6.1)  P  =  (x  >  0  and  integral  : 

where  r  is  a  nondecreasing  integral 
A  vector  function  Z  =  RN  -*  F+ 

(6.2)  P-(v)  >  p^(u)  >  0  for  all 

where  p^(v)  =  Z(v+e^)  -  Z(v)  and 
indexed  by  i  £  N  is  equal  to  1. 

Given  a  nondecreasing  submodular 
of  problem  (1.1)  is 


I  x.  <  r(S),  VS  c  N} 
j£S  J 

sutmodular  set  function  with  r(0) =  0. 
is  submodular  and  ncndecreasing  if 

i  £  N  and  v  <  u  £  f\ 

e.  is  the  unit  vector  whose  component 

l 

vector  function  Z,  a  generalization 


\ 


(6.3) 


max  (Z(v)  :  v  €  X) 


where  X  is  the  intersection  of  p  integral  polyr.atroids.  Note  that,  as  a 
consequence  of  (6.1),  the  problem  (6.3)  can  be  written  as 

(6.4)  max  Z(X) 

Ax  <  b 

x  >  0  and  integral. 


where  A  is  a  0,1  matrix.  Conversely,  for  any  0,1  matrix  A,  the  problem 

(6.4)  is  equivalent  to  (6.3)  where  X  is  the  intersection  of  a  nurber  of 

integral  polymatroids .  For  example  X  =  Ci  p.  where  P.  =  {x  >  0  and 

f.  i  1  1  rh 

integral  :  a.  x  <  b.  }  ,  a.  is  the  icn  row  of  A  and  b.  is  the  ic  cor 

ill  l 

of  b  . 


I'rponent 


A  steepest  ascent  (or  greedy)  algorithm  for  solving  problem  (6.3)  or 

(6.4)  would  be 

Greedy  alycrii'hm 

Initialization  :  Set  v°  =  0  and  t  =  1. 

•  Find  jt  (  R  such  that  p.  (vC  *)  =  max  {p-Cv*-  *)  : 

J «  J 

t-1  L 

v  +  e^  6  X} .  If  no  such  j ^  exists,  stop. 

Otherwise  set  vC  =  vC  '  +  e.  ,  increment  t  by  1 

Jt 

and  repeat  Step  t. 

In  this  section  vce  assume  Z(0)  =  0.  If  we  define  =  p.  (v£  *), 

k  G  ^ 

the  value  of  the  greedy  solution  v  is  Z  =  pt  ♦  ...  ♦  p^,  where  k  is 
the  value  of  the  parameter  t  when  the  greedy  algorithm  stops.  In  fact, 
k  =  r(N)  as  defined  in  (6.1).  Note  that  the  greedy  algorithm  defined 
above  is  not  polynomial  in  ]n|. 


Let  m  =  (m.  :  i  €  N)  where  is  the  largest  integer  X  such  that 
the  vector  1  e.  €  X.  Define  the  total  curvature  of  Z  with  respect  to  X 


-30 


a  =  max  < 
m*  l 


P  •  (0)  -  p .  (m  -  e . ) 


Pj  (0) 


-■here  N*  =  {j  £  N  :  p ^  (0)  >0}. 


ThECREM  6.1.  Let  X  be  the  intersection  of  p  integral  pel  gnat  raids 

i  =  and.  Z  a  nondecreasing  submodular  vector  function  with  Z(0)  =  0 

and  total  curvature  a.  Then  a  greedy  solution  to  problem  (6.3)  lias  a  value 
G  1  #  ^ 

Z  >  —  —  Z  where  Z*  is  the  optimal  value. 

Proof  :  Consider  an  optimal  solution  u)  .  We  will  write  a)  =  S  where 

(£)  (£)  £=1 

the  e  s  are  unit  vectors,  i.e.  e  =  e.^.  for  some  i(£)  €  N.  Note 

that  the  same  unit  vector  e^  may  appear  several  times  in  the  summation, 

indexed  by  different  values  of  i . 

Let  sl  be  the  vector  obtained  at  iteration  t  of  the  greedy  algorithm, 
t  =  I , . . .  ,k.  If  | oj |  >  t,  then  we  claim  that  ,  for  all  i, 

(6.5)  s1  +  eU)  €  P.  for  at  least  j to |  -  t  of  the  vectors  . 

This  is  proved  by  repeated  use  of  axiom  (ii)  of  the  definition  of  inte¬ 
gral  polymatroids  :  consider  to'  <  ca  such  that  |u)' |  =  t+1.  By  (ii), 

(£)  t  t  m  •  t  (?) 

3  e  such  that  s  <s  ♦  e  <  a)  vs.  Now  replace  to  by  to  -  ’* 

and  repeat  the  argument.  Since  it  can  be  repeated  |uj|  -t  tines,  the  proof 

of  the  claim  (6.5)  is  complete.  A  consequence  of  property  (6.5)  is  that 

(6.6)  if  J  to  J  >  p  t ,  then  st+e^^£  n  P.  for  at  least  |u)J  -  pt  of  the  vectors  e^ 


*r°r  any  such  Pj(£j  (sC)  <  Pc+jt  as  a  consequence  of  the  choice  made 
by  the  greedy  algorithm..  So  the  e^^’s  can  be  ordered  so  that 
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(6.7)  p.  (£)  ^  Pt+1  for  pt  ^  *■  ^  p(t+1)  and  t  -  0, . . .  ,k-l . 

lu)  I  -  . .  , . 

Note  that  k> -  as  a  consequence  of  (o.o). 

P 

,  k 

Let  uj  a  s  be  the  vector  whose  component  is  nin  (u)j  ,  s  ^  )  and  let 

k  (£) 

L  C  {1.....U}  be  a  set  of  indices  such  that  d  a  s  =  X  e  .  Then 
-  L 


2(u)*(l-a)  X  Pf  <Z(uvsV  X  p  (sk)  <  Z(sk)  ♦  X  p..  fsk). 

t-1  C  ££L  lW  £=  1  lU) 


As  a  consequence  of  (3.7) 

k  k 

Z(u>)  +  (1  -a)  X  p  <  (1  +p)  2  p  . 

t=l  t=l  C 

♦  G 

Therefore  Z  <  (p  + a)  Z  as  required. 


□ 
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APPENDIX  :  proof  of  Theorem  4.5 


PROPOSITION  :  Let  t  1  <  n  <  K ,  be  given  in  (4.15).  •  The  difference 
Xpn+j  -  (A  -  1)  pn>  1  <  n  <2h  -  K,  is  equal  to 


Proof . 


Xpn+,  - 


A(K 

A(K 


>)  -  (A  -  J)  MK  - 

„n+l 


\)_z  R(A  - 1) 

K  '°n 


K_- A 
K 


P 


n 


□ 


We  are  now  ready  for  the  proof  of  the  theorem  :  We  prove 


Pt(s)  2?  P  <R>  >  0,  for  each  S  C  R  C  E,  for  each  f.  €  E. 


Ca.sc  1  :  t  €  B 


l.a) 


j<K-h  (note  that  r  =  0,  q  -  j) 


Djl(B1  u  WJ  u  a‘D)  =  p.  +  j  +  (j  ^!t)Di  +  2  -  (j  4^)Pi  +  ] 

Pt  =  (j  +  m)Pi  +  2  "  4  ra  ~  OPi+r 


From  the  above  proposition,  by  substituting  j +  m  for  A,  we  get 


p£  = 


_  K  -  (j  +  m) 


K 


*r 


• . -  -*£-■** 
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Since  j  <  K  -  h,  m  <  h  ,  j  +  m  <  K,  ve  have  p^  >  0.  Moreover ,  by  1  over  5  ng 
j  or  m  or  i,  cannot  decrease. 


l.b)  i+j<h,  j  >  K  -  h  (note  that  q  -  K  -  h ,  r  =  j  +  (K  -h)  >  0) 

P£(Bl  U  W3  U  A">  =  Pi+r+,  +  (K  "  h+  m)Di+r+2  -  (K  "h  +  ^Pi+r+l 

=  (K  -h  +  rc)pi  +  r  +  2  “  (K  -h  -  1  +  "0P;+r+1  - 


Again,  from  the  preceding  proposition,  by  setting  X  =  (K  -h  + m) ,  we 


get  : 


h  -  in 

Pl  ~  ~K~  Pi+r+] ' 


Since  Pj  +  r+j  ^  0  and  m  >  we  get  p^  >  0.  Moreover,  if  i  or  r  or  m 


decreases,  p^  increases  and 


h  -  m  K  -  ( j  +  m)  ,  .  ^  . 

~  Pi  +  r+l  < - K -  Pi+1  vhen  J  <  K  ~h  • 


4 


Pi-*r+l  ls  i'reater  ur  equal  to  p,h_},+  ]  becsuse  when  r  ^  0,  i  <  2h-v; 

v!,en  r>0,  r  =  j  -  (K-h),  but  i  -»  r  =  i  +  j  -  (K  -h)  <  h  -  (K-h)  =  2h-K. 


flU  C  3  :  £  €  Y  U  T 

3. a)  i  +  j<h,  j  <  K  - h  (note  that  r  --  0,  q  -  j) 

V*'  UWj  u  a")  .  (j.l  .rtpi(]  -  =  ci>r 

j  >  K  -  h  (note  that  q-K-h,  r>0) 

Pt(B‘  u  WJ  u  A")  =  Pi<r<1  *  (K  -h  -  <K-b4„)oi(Ml 

-  <K-h4.)0.,r4j  -  (K-b-1  *«)0.<r4|. 


Again,  by  applying  the  proposition,  we  get 

„  _  h-“ 

P£  K  Pi+r+l 

Smoe  ~<1  and  Pi+r+,  <  P-  +  ,  the  condition  P.  +  ]>  ---  Pi  +  r+, 
always  satisfied. 

3.c)  i  +  j  >  h 

P£(b1  U  Wj  u  Am)  =  0. 


Therefore,  the  function  is  subnodular  and  nondecr easing. 
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For  the  problem  max(Z(S)  :  S  is  an  independent__set  in  the  matroid  X),  it  is  ) 

well-known  that  the  greedy  algorithm  finds  an  optimal  solution  when  Z  is  an  j 

additive  set  function  (Rado-Edmonds  theorem).  Fisher,  Nemhauser  and  Wolsev  j 
have  shown  that,  when  Z  is  a  nondecreasing  submodular  set  function  satisfying  • 
Z(4)  •  0,  the  greedy  algorithm  finds  a  solution  with  value  at  least  half  the  J 
optimum  value.  In  this  paper  we  show  that  it  finds  a  solution  with  value  at  j 
j  least  1/(1  +  a)  times  the  optimum  value,  where  a  is  a  parameter  which  repre- 
[  gents  the  "total  curvature”  of  Z.  ThTs~TFhrainecdr  Satisfies  G  ■* Tt  -»  1  and 


00  ,:Sn  1473  toinew  of  i  nov  •»  is  omolcti  (continued) 

S/N  0  tO  J*0|4«  1101  _  .  .  _ 
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a  »  0  if  and  only  if  Che  sec  function  Z  is  additive.  Thus  the  theorems  of  Rado- 
Edmonds  theorem  is  given  in  terms  of  a  "greedy  curvature"  of  the  set  function. 

Unlike  the  first  bound,  this  bound  can  prove  the  optimality  of  the  greedy  algorithm 
even  in  instances  where  Z  io  not  additive.  A  third  bound,  in  terms  of  the  rand  and 
the  girth  of  X,  unifies  and  generalizes  the  bounds  (e— l)/e  known  for  uniform  matroids 
and  1/2  for  general  matroids.  We  also  analyze  the  performance  of  the  greedy  algorithm 
when  X  is  an  independence  system  instead  of  a  matroid.  Then  we  derive  two  bounds,  both 
tight:  . 

The  first  one  is  [1  -  (1  -  n/K)  ]/a  where  K  and  k  are  the  sizes  of  the  largest  and 
smallest  maximal  independent  sets  of  X  respectively;  the  second  one  is  l/(p  +  a) 
where  p  is  the  minimum  number  of  matroids  that  must  be  intersected  to  obtain  X. 


